00100				RESEARCH PLANS
00200			VISION, MANIPULATION, PRODUCTIVITY
00300				T. O. Binford
00400	
00500	This is a description of plans for the hand/eye group for the next few years in vision,
00600	manipulation, and automation.
00700	
00800	The general program is to explore important issues of artificial
00900	intelligence within the domain of intelligence which interacts
01000	with the real world.  The predominant concern is representation.
01100	In the past, I have been concerned with representation of shape
01200	and space.  Now we are broadening our concern to representation
01300	of other visual constructs, strategies and visual knowledge,
01400	to descriptions of assemblies.  I have a strong interest in
01500	perception of biological systems.  My interest is not in
01600	imitating these systems; not enough is known to even try.
01700	Instead, the approach has been to understand the problems
01800	faced by perceptual systems, natural or artificial, and to
01900	arrive at solutions to those problems which are not dead-ends.
     

00100			REPRESENTATION
00200	
00300	My interest is "representation of what and for what?".  It turns out that
00400	there are significant applications for our research in
00500	representation.  This may seem strange that the more abstract
00600	parts of our effort have application, but language is closely
00700	dependent upon representation (productions for syntax, formal
00800	semantics) and language is essential for control and computerization
00900	of industry.
01000	
01100	My representation of object shape by primitives defined by "generalized
01200	translation invariance" has been the basis of a continuing program of
01300	of description and recognition of complex objects [Agin, Nevatia].
01400	The product of their work has been the recognition of objects of the
01500	complexity of a doll or a toy horse.  The representations are used
01600	in research on assembly strategies by Taylor.  A special case of the
01700	class of representations is used by Hollerbach at MIT.
01800	
01900	There seem to be two streams of work on representation.  One of these
02000	concerns itself with an embedding framework for representing general
02100	knowledge.  The other is concerned with representing the semantics of
02200	particular domains.  For example, to prove theorems in geometry,
02300	we use results from geometry; we do not go back to formal logic.  I.e.,
02400	logicians don't prove fundamental theorems in algebraic geometry.
02500	But they do contribute to the foundations of other fields of mathematics.
02600	Thus, the power of geometry comes from the representing of geometry
02700	(domain-specific representation) and not predominantly from
02800	the general system.  We have need for many domain-specific
02900	representations.
03000	
03100	I do not think that we can build large intelligent perceptual
03200	systems without extensive strategy and high level elements, nor
03300	can we make those elements without well-defined representations
03400	which are intimately connected to semantics of the world and the
03500	problem domain.
     

00100			RESEARCH IN MANIPULATION
00200	We are building on an extensive system built up by Paul, based
00300	on Scheinman's manipulator.  The total system is impressive,
00400	combining arm servo control, trajectory planning, a language WAVE
00500	at the level of assembler code, and macros for frequently used
00600	operations.  Several demonstration and assembly tasks have been
00700	performed by Paul and Bolles [].  A great deal has been done with
00800	touch and force sensing which is quite crude.  Shimano has implemented
00900	force sensing hardware and software about an order of magnitude more
01000	sensitive than previous.  Takeda intends to implement and experiment
01100	with sensitive touch sensors with better spatial resolution.  A
01200	broad range of possibilities is opened with these enhanced sensory
01300	capabilities:  formulating the coordinated action of two arms (carrying an object,
01400	using a wrench), or formulating the concepts in carrying out new
01500	assembly tasks.
01600	
01700	The hand language was extremely valuable; it led to a generality of
01800	the sort that once we had performed an assembly like the pump, carrying
01900	out assembly of a similar device would require only a day or two.
02000	Paul, Finkel, Bolles, and Taylor are wrapping up the design of the first
02100	level of a new hand language HAL, and have begun implementation.  The
02200	new language is high level with an ALGOL like form.  
02300	It also is model-based, with some automatic strategy generation
02400	Coordinated two-arm motion and other new manipulation abilities
02500	will be included.
02600	
02700	There are a number of potential research areas in manipulation
02800	for which no one is committed.  These include:
02900		obstacle avoidance
03000		redundant degrees of freedom (for obstacle avoidance)
03100		three finger hand
03200		real-time trajectory calculation
03300		high performance control.
     

00100			VISUAL STRATEGIES
00200	There are several typical problems for visual research:
00300	1. Repetitive problems for which the two-dimensional scene
00400	does not change very much.  Visual feedback problems are typical
00500	of this case.  In this case, we can make a two dimensional model.
00600	2. Few objects, simple, well-known environment, with no unknown objects.  
00700	An example is the office scene of SRI.  In this case, strategies
00800	can proceed with very simple pointwise tests "the telephone is
00900	the only black object".
01000	3. Simple environment, more objects and unknown objects.  This is
01100	more typical of a usual office.  In this case, we cannot verify
01200	object identifications by simple pointwise operations, but must use
01300	shape.
01400	4.  A relatively complex visual environment with many objects.
01500	Here, our representations must be three-dimensional.  Current low-level
01600	description mechanisms are probably inadequate here.
01700	
01800	One student, Garvey of SRI, is finishing a thesis concerning the
01900	automatic generation of programs to identify objects in simple
02000	office scenes.  That work is based on point properties.
02100	Point properties are enormously simpler than those dealing with
02200	shape.  They are much weaker, however, and we cannot expect to cope
02300	with much more than "telephones are the only black object in the
02400	scene".  We will be using shape with visual strategies, and visual
02500	strategies directed to complex objects.
02600	
02700	It is difficult for humans to make vision programs in LISP or SAIL,
02800	and to make possible visual strategies, a vision language is useful.
02900	The vision language is essentially a coding of the representations
03000	for external constructs (2d and 3d scene) and internal semantic
03100	descriptions (edge fragments, curves, relation graphs).
03200		VISUAL FEEDBACK
03300	
03400	This domain of research raises the problems of how we represent
03500	visual knowledge in a domain for which we have a lot of detailed
03600	knowledge, and how do we represent strategies for a broad class
03700	of examples, so that programming a special case is quite simple.
03800	Visual feedback tasks are repetitive, and for each one, we have
03900	access to a good model of the expected visual environment. 
     

00100	cooperation on hardware--sensor cooperation
00200	vision language--cooperation on software
00300	strategies for vision and assembly
00400	visual feedback
00500	planetary exploration
00600	stereo
00700	integrated visual systems
00800	software plans
00900	
01000	manipulation implies some form of depth perception.
01100	strategies imply the need for intermediate level of
01200		language, ie vision language
01300	representation is the key to visual systems
01400		useful representations at all levels are to
01500		be included in the vision language
01600	
01700	
01800	SOLID STATE IMAGE DEVICE INPUT
01900	GE will start delivery in December of 188x244 CID (charge injection device)
02000	solid state image sensors.  These have the following properties of interest
02100	to us:
02200		no blooming
02300		better signal to noise ratio than TV
02400		variable input rate (making easy to match to PDP11 bus)
02500		potential random access
02600		very small
02700		small optics
02800	In implementing such a system, we are negotiating a joint effort with
02900	several other labs to develop a single, plug-in device including
03000		pan,tilt,stereo,color filters, zoom or fovea
03100	The standard device would presumably accomodate several different
03200	basic solid-state sensors, interchangeable.  The largest interface
03300	problem at this point is the mechanical interface.
03400	One possibility for the mechanical system is to use the small
03500	Scheinman arm for the pan/tilt system.  For the first two or three
03600	joints of that arm, the cost would be about $2300 to Scheinman,
03700	without manual controller.  We would need motorized zoom,iris,
03800	focus,filters and stereo.  For those, we can find part of them
03900	available in super-8 movie equipment.  Clearly, the whole of such
04000	a system will cost us about $5000 for the pan/tilt and $3000 for
04100	the zoom,iris, etc.  Having a standard version of such a system
04200	would make it possible for other laboratories to immediately use
04300	these devices.
04400	
04500	These plans bear upon the problem of how to make the PDP11 system
04600	useful for vision.  As we all know, the hardware building for the
04700	PDP11 has gone very slowly, because the memory map, single bit
04800	error correction and memory interface are complex.  Part of the
04900	complexity is the 72 bit wide channel for visual input and display
05000	output.  The bandwidth requirements of the solid state device are
05100	low enough to work on the bus.  If we could get a  low bandwidth
05200	color display device, then there would be no motivation for
05300	extending the memory bandwidth.